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ABSTRACT 

The  Basket  Method  and  Dollar  Unit  Sampling  techniques  are  examined  to 
determine  which  of  them  draws  a  sample  which  allows  a  more  accurate  estimate  of  a 
population  to  be  made.  The  several  populations  used  in  the  simulation  have  errors 
planted  to  represent  both  random  "honest"  mistakes  and  weighted  "dishonest" 
mistakes.  The  author  concludes  that  the  Basket  Method  has  a  more  desireable 
accuracy  pattern  than  the  Dollar  Unit  Sampling  Technique. 
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I.  INTRODUCTION 

A.       NEED  FOR  THIS  STUDY 

In  major  weapon  system  acquisition,  the  Department  of  Defense  typically  issues 
a  large  number  of  change  orders  to  modify  an  existing  contract.  In  each  case,  the 
contractor  prepares  a  proposal  of  contract  price  adjustment  for  the  requested  changes. 
A  lead  ship  or  aircraft  production  contract  may  generate  over  10,000  such  proposals. 

Acquisition  regulations  require  the  government  to  analyze  each  proposal  and 
negotiate  with  the  contractor  for  a  fair  price  adjustment.  However,  these  requirements 
produce  more  work  than  the  existing  number  of  government  auditors  can  undertake. 
Generally,  as  backlogs  develop,  thoroughness  is  reduced  for  expediency,  and  errors  slip 
by.  However,  work  in  progress  usually  goes  on,  but  contractors  are  not  paid  until  after 
the  proposal  is  processed.  This  delay  in  funding  raises  the  capital  cost  to  the 
contractor  for  performing  work  for  the  government. 

It  is  in  the  best  interests  of  both  the  government  and  the  contractors  to  raise  the 
accuracy  and  speed  and  reduce  the  cost  of  auditing  such  proposals  of  price  adjustment. 
If  it  were  allowed  by  the  acquisition  process,  analyzing  and  negotiating  a  sample  of 
proposals  selected  with  an  effective  sampling  technique  could  have  these  effects. 

The  reason  for  auditing  the  proposal  population  is  to  ensure  the  proposals  are 
realistic  and  fair.  During  an  audit,  a  proposal  will  be  found  to  be  understated,  that  is 
the  Government  feels  that  the  proposed  cost  of  a  change  to  the  contract  is  less  than 
the  actual  cost  the  contractor  will  incur,  or  overstated,  or  correct.  Any  overstatement 
or  understatement  is  considered  an  error  in  the  proposal  population.  The  ideal 
sampling  technique  would  choose  a  sample  from  the  population  of  proposals  which, 
when  audited,  would  always  give  an  estimated  value  for  the  population  as  a  whole 
which  was  exactly  correct,  no  matter  what  the  degree  or  distribution  of  errors  in  the 
proposal  population.   Such  a  technique  does  not  exist. 

Assuming,  then,  that  there  will  be  some  degree  of  error  in  the  prediction  of  the 
true  value  of  the  entire  proposal  population  whenever  a  sampling  technique  is  used;  the 
behavior  of  the  degree  of  error  must  be  predictable  and  exhibit  certain  qualities  in 
order  for  the  sampling  technique  to  be  considered  appropriate  for  the  purpose 
described  above. 


Specifically,  the  degree  of  error  should  not  be  easily  altered  by  the  error 
arrangement,  that  is  the  distribution,  size,  frequency,  or  type  (overstatement  or 
understatement)  of  errors  found  in  the  population,  as  long  as  the  total  error  is  the 
same.  A  sampling  procedure  which  shows  little  response  to  changes  in  the  error 
arrangement  will  be  said  to  show  consistency.  If  certain  patterns  of  errors  caused  the 
entire  population  to  be  evaluated  as  understated  then  the  government  would  pay  more 
than  a  fair  price  for  the  changes  described  by  the  population  of  proposals  which 
contained  those  errors.  This,  in  turn,  would  tend  to  encourage  fraud,  since  a 
contractor  could  carefully  seed  his  proposal  population  with  deliberate  errors  of  the 
appropriate  size,  type,  and  distribution  and  thereby  be  awarded  a  larger  payment  from 
the  government. 

Therefore,  the  desired  method  will  not  necessarily  be  the  one  which  results  in  the 
most  accurate,  average  estimate  for  various  error  arrangements  in  the  proposal 
population.  It  will  instead  be  the  method  which  responds  least  to  variations  in  the 
error  arrangement. 

B.       METHODOLOGY 

In  this  paper  the  Dollar  Unit  Sampling  Method  and  the  Basket  Method  will  be 
used  to  draw  samples  from  populations  for  evaluation.  The  populations  were  used 
earlier  in  a  joint  study  of  the  American  Institute  of  Certified  Public  Accountants  and 
the  American  Statistical  Association.  Each  population  consists  of  two  columns  of 
values  which  represent  the  proposed,  or  book  value  of  a  contract  change  and  the 
audited  or  true  value  of  that  change.  The  populations  are  rigged  with  either  random  or 
planned  errors.  Samples  drawn  by  the  two  methods  from  each  population  will  be 
evaluated  and  compared  to  determine  which  method  gives  a  better  estimate  of  the 
whole  population  according  to  the  goals  described  above.  Both  the  error  rigging  and 
evaluation  steps  are  explained  further  in  the  description  of  the  simulation. 

The  amount  of  work  associated  with  auditing  is  more  closely  related  to  the 
number  of  items  being  audited  than  to  the  total  dollar  value  of  all  the  items  being 
audited.  Therefore,  the  sampling  rules  of  the  two  methods  will  be  adjusted  so  that  they 
will  draw  samples  with  the  same  number  of  proposals  from  each  population.  The 
results  of  this  study  will  then  indicate  which  method  yields  the  more  desirable 
prediction,  that  is,  the  greater  benefit,  while  holding  the  cost  of  the  audit  constant. 


II.  DOLLAR  UNIT  SAMPLING 

A.  HISTORY 

Dollar  Unit  Sampling  is  a  technique  which  is  strongly  based  on  statistical  theory. 
It  determines  the  possible  error  in  a  sample  based  on  a  Poisson  model  of  the 
population. 

The  roots  of  the  Dollar  Unit  Sampling  method  can  be  traced  to  the  1960's  when 
van  Heerden  developed  the  "guilder  number"  method  [Ref.  1].  About  the  same  time, 
K.  W.  Stringer  developed  a  similar  model  which  he  called  Cumulative  Monetary 
Amounts  (CMA)  Sampling  [Ref.  2].  Dr.  Albert  Teitlebaum  then  developed  a  model 
similar  to  Cumulative  Monetary  Amounts  Sampling  which  he  called  Dollar  Unit 
Sampling  (DUS)  [Refs.  3,4,5,6].  In  December  of  1973  he  presented  the  first  attempt  at 
a  comprehensive  proof  for  Dollar  Unit  Sampling,  which  led  to  rising  prominence  of  the 
Dollar  Unit  Sampling  method  [Refs.  3,7]. 

Dollar  Unit  Sampling  is  also  known  as  Monetary  Sampling  (MS),  Combined 
Attribute  Variable  Sampling  (CAV),  Monetary  Unit  Sampling  Technique  (MUST), 
Sampling  Proportionate  to  size  (SPS),  and  Probability  Proportional  to  Size  (PPS) 
[Ref.  8]. 

The  CMA  procedure  is  the  most  widely  used  version  of  DUS,  but  it  has  the 
disadvantage  of  overestimating  the  possible  overstatement  [Refs.  7,8,9].  While  this  is  a 
conservative  error,  from  the  government's  viewpoint,  it  is  still  an  error,  and  contractors 
will  be  unlikly  to  bind  themselves  to  a  method  which  will  routinely  overcorrect  against 
them. 

B.  DESCRIPTION  OF  DOLLAR  UNIT  SAMPLING 

There  are  three  steps  to  using  the  Dollar  Unit  Sampling  method:  calculating  the 
sample  size,  choosing  the  sample,  and  calculating  the  error.  These  steps  will  be  briefly 
discussed  below. 

1 .  Calculating  Sample  Size 

There  are  four  commonly  used  methods  for  the  determination  of  the  sample 
size.  All  require  the  total  book  value  of  the  contract  changes,  the  reliability  factor  for 
a  given  level  of  confidence,  and  the  amount  of  error  which  can  be  tolerated  for  a  given 
level  of  confidence.    The  better  the  estimated  amount  of  error  in  the  sample  is  known, 


the  more  accurate  the  required  sample  size  will  be.  In  practice,  tables  are  used  which 
specify  the  sample  size  based  certain  arguments  for  book  value,  level  of  confidence,  and 
estimated  error  rate. 

In  this  study;  however,  the  process  of  selecting  a  sample  size  is  not  required  to 
be  followed.  This  is  because  in  actual  applications  where  the  true  value  of  the 
population  is  not  known  the  only  way  to  be  reasonably  certain  that  one's  results  are 
valid  is  by  complying  with  rules  which  will  tie  the  audit  to  statistical  theory.  The 
sampling  rules  for  Dollar  Unit  Sampling  are  intended  to  do  just  that,  so  that  the 
auditor  who  follows  the  sampling  procedures  will  be  able  to  determine  the  extent  of  the 
audit  required  to  achieve  the  desired  certainty. 

In  this  study  the  true  values  of  the  proposals  are  known,  as  are  the  size  and 
distribution  of  the  errors,  and  the  Dollar  Unit  Sampling  method  is  not  being  compared 
to  its  theoretical  limits,  but  to  a  second  method  to  determine  which  of  the  two  gives  a 
more  desireable  result  in  a  certain  case  for  the  same  amount  of  work.  Therefore,  the 
size  of  the  sample  must  be  based  on  a  number  of  proposals  rather  than  a  number  of 
dollars.  This  being  the  case,  one  tenth  of  the  total  number  of  proposals  in  the 
population  was  arbitrarily  chosen  as  the  size. 
2.  Drawing  the  Sample 

The  heart  of  Dollar  Unit  Sampling  is  the  choosing  of  the  sample.  Rather  than 
a  random  technique,  which  might  select  a  certain  number  of  contracts  to  audit,  the 
sample  size  calculated  in  the  previous  step  refers  to  a  number  of  single  dollar  units. 
Single  dollar  units  are  merely  dollars.  Thus,  a  proposal  which  is  estimated  at  one 
hundred  dollars  would  contain  one  hundred  single  dollar  units.  Each  of  these  dollars 
has  the  same  probability  of  being  selected  as  every  other  dollar  in  the  entire  proposal 
population.  When  a  dollar  unit  is  selected,  the  entire  proposal  in  which  it  resides  is 
audited.  Since  the  single  dollar  units  are  each  given  the  same  chance  of  being  selected, 
a  proposal  with  a  larger  dollar  value  than  another  has  a  larger  chance  of  being  selected 
for  the  audit. 

To  select  the  dollars,  and  thereby  the  proposals,  to  be  audited,  the  proposals 
are  stacked  and  then  added  to  form  a  series  of  partial  sums.  The  proposals  which  lie 
between  the  partial  sums  which  bracket  the  random  numbers  generated  by  the  sizing 
calculations  are  selected  for  auditing.  A  simple  algorithm  for  accomplishing  this  is  to 
subtract  the  proposal  book  values  from  the  random  number  until  the  difference  is 
negative  and  audit  the  last  proposal  subtracted.  The  step  by  step  process  is  as  follows: 
1.      Arrange  the  proposals  in  a  sequence  and  find  the  sum  of  their  book  values. 


2.  Calculate  the  Average  Sample  Interval  (ASI)  using  the  equation: 

ASI  =  B/n 

where 

B  =  sum  of  book  values  and 

n  =  number  of  dollars  to  be  chosen  as  determined  by  the  sample  size  calculation 

3.  Select  a  random  start  number  RS  such  that  0  <  RS  <  ASI. 

4.  Subtract  the  proposal  book  values,  in  the  sequence  determined  in  step  one, 
from  RS  until  the  difference  is  negative. 

5.  Audit  the  proposal  which  makes  the  running  difference  negative,  add  the  ASI  to 
the  negative  difference,  and  repeat  from  step  4. 

If  a  proposal  has  a  book  value  greater  than  the  ASI  it  will  be  chosen.  A 
proposal  is  only  allowed  to  be  chosen  once,  no  matter  how  large  it  is,  so  the  number  of 
dollar  units  calculated  when  determining  the  sample  size  will  be  greater  than  number  of 
proposals  selected  for  audit  if  any  proposals  have  a  book  value  greater  than  the  ASI. 
The  development  of  digital  computers  has  greatly  eased  the  execution  of  this  phase. 
3.  The  Peach  Problem 

Before  discussing  the  evaluation  phase,  it  is  useful  to  briefly  illustrate  and 
develop  the  Dollar  Unit  Sampling  technique  by  examining  the  'Peach  Problem'  [Ref.  8]. 
In  the  Peach  Problem  one  must  determine  the  portion  of  peaches  in  a  lot  of  one 
million  which  is  unusable.  To  determine  the  size  of  the  portion  one  will  draw  a  sample 
and  use  information  on  both  the  number  of  rotten  peaches  and  the  extent  of  rottenness 
found. 

One  hundred  sample  peaches  are  randomly  drawn  and  examined.  If  there  are 
no  defective  peaches,  there  is  a  5%  risk  that  the  percentage  of  peaches  defective 
exceeds  3%.  These  numbers  are  based  on  the  Poisson  distribution  which  is  chosen 
because  it  is  a  good  estimater  of  the  binomial  case  being  examined.  The  binomial  is 
that  a  peach  is  either  rotten  to  some  detectable  degree  or  it  is  not.  Similarly,  a 
proposal  is  either  overstated  or  understated  to  some  significant  degree  or  it  is  accurate. 
It  should  also  be  noted  that  the  terms  mentioned  in  the  sample  size  calculation  now 
take  on  values.  The  total  book  value  is  one  million.  The  a  risk,  or  risk  of  accepting 
an  unacceptable  lot,  is  5%,  which  means  the  confidence  is  95%.  The  reliability  factor 
for  the  95%  confidence  level  is  3%,  meaning  that  at  most,  30,000  peaches  could  be 
rotten  to  some  degree  and  the  population  would  still  be  acceptable. 

If  there  had  been  one  defective  peach,  there  would  have  been  a  5%  a  risk  that 
the  percentage  of  defective  peaches  exceeded  4.75%.    Similarly,  if  two  defective  peaches 
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were  found,  there  would  have  been  a  5%  a  risk  that  the  number  of  defective  peaches 
will  exceed  6.31%.   These  values  also  come  from  the  Poisson  distribution. 

What  is  not  considered  yet  is  the  fraction  of  the  defective  sample  peaches 
which  is  rotten.   This  is  important  if  one  can  use  a  partial  peach  or  partial  dollar. 

To  account  for  this  in  the  case  with  two  defective  peaches  in  the  sample,  the 
two  defective  peaches  are  examined.  Assume  the  two  rotten  peaches  were  8%  and  4% 
rotten.   Then  in  95%  of  the  cases: 

1.  The  number  of  totally  rotten  peaches  will  be  less  than  3%. 

2.  The  number  of  8%  rotten  peaches  will  not  exceed  an  additional  1.75%.   (4.75% 
-  3%) 

3.  The  number  of  4%  rotten  peaches  will  not  exceed  1.56%.   (6.31%  -  4.75%) 
Then  the  total  portion  of  the  population  which  is  rotten  will  be  less  than: 

3.00%  x  100%  =  3.00% 

1.75%  x  8%  =  0.14% 

1.56%  x  4%  =  0.06% 

4.62% 

This  is  a  better  estimate  of  the  total  amount  of  rotten  peaches  in  the  lot  than 
6.31%  because  it  corrects  for  the  usable  part  of  the  partially  rotten  or  'tainted' 
peaches.  It  should  be  noticed  that  the  additional  corrections  are  arranged  in  order  of 
decreasing  percent  tainting.  This  gives  the  most  conservative  result  since  the  smaller 
percent  taintings  are  applied  to  the  smaller  percent  defect  increments. 

The  analogy  applies  directly  to  the  proposals  problem.  The  peaches  are 
dollars,  each  one  essentially  the  same  size.  The  degree  of  rottenness  is  the  degree  of 
error  in  the  entire  proposal  in  which  the  sampled  dollar  is  found.  Thus,  if  a  chosen 
dollar  is  in  a  proposal  which,  when  audited,  reveals  an  8%  overstatement  error,  each 
dollar  in  that  proposal,  and  in  particular,  the  dollar  which  was  chosen  to  be  sampled,  is 
considered  8%  wasted. 

4.  Calculation  of  the  Result 

If  there  are  no  errors  in  the  sample,  no  calculation  is  required  and  the  amount 
of  error  for  the  planned  level  of  confidence  is  the  planned,  tolerable  amount  of  error. 

Similarly,  if  the  sample  size  was  determined  using  the  p  Risk  Only  tables,  and 
the  sum  of  the  taintings  in  the  sample  is  less  than  that  planned  when  the  sample  size 
was  determined,  then  the  resulting  tolerable  amount  of  error  is  less  than  that  planned 
and  no  calculation  is  required. 
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If  the  sample  size  was  determined  using  the  a  and  P  risk  tables,  and  the  sum 
of  the  taintings  is  less  than  the  K-factor,1  then  the  actual  error  is  less  than  the  planned, 
tolerable  error  limit;  three  percent  in  the  previous  example. 

If  some  other  result  is  reached  then  some  calculation  will  be  necessary.  Each 
erroneous  proposal  is  listed  in  order  of  decreasing  tainting.  The  taintings  are  then 
multiplied  by  the  incremental  percent  defect.  The  sum  of  the  taintings  and  tainting 
products  are  then  used  to  compute  the  net  upper  error  limits. 

If  any  errors  are  found  materiality  must  be  redefined  since  the  sample  size 
anticipates  that  the  sample  will  contain  no  errors.  If  the  errors  found  are  few  or  small, 
the  new  materiality  might  be  close  enough  to  the  original  goal  to  allow  acceptance  of 
the  population. 


The  K-factor  is  one  of  the  two  values  read  from  the  tables  used  to  determine  the 
sample  size,  the  other  being  the  sample  size.  The  K-factor  is  significant  because  it  is 
compared  to  the  sum  of  the  tainting  in  the  sample  to  determine  if  the  population  can 
be  accepted  without  exceeding  the  P  risk  (]T  taintings  ^  K-factor)  or  rejected  without 
exceeding  the  a  risk.  (£  taintings  >  K-factor) 
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III.  THE  BASKET  METHOD 

A.  HISTORY 

The  Basket  Method  was  developed  by  Dr.  K.  T.  Wallenius  under  the  sponsorship 
of  the  Office  of  Naval  Research  and  the  Naval  Material  Command.  It  is  a  recently 
created  technique  which  was  formed  in  direct  response  to  the  problems  of  auditing  and 
negotiating  numerous  contract  proposals  with  sole  source  contractors  [Ref.  10]. 

B.  DESCRIPTION 

The  Basket  Method  derives  its  name  from  its  process  of  sorting  a  population  into 
a  number  of  baskets,  one  of  which  will  be  randomly  selected  for  sampling.  The  aim  of 
the  basket  assignment  process  is  to  make  each  basket  resemble  the  population  in  any 
traits  of  interest.  In  general,  the  spread  and  proportion  of  proposal  values  will  be 
nearly  identical  to  the  population.  Sophisticated  software  can  quickly  balance  baskets 
for  work  center  assignments,  material  usage,  skill  requirements,  etc.  In  short,  whatever 
trait  is  deemed  pertinent  to  the  value  of  an  audit  will  be  balanced  by  the  Basket 
Method  where  possible.  Obviously,  in  a  population  of  proposals  where  one  is  ten  times 
larger  than  any  others  it  would  be  more  practical  to  remove  that  proposal  from  the 
population  before  assigning  the  baskets. 

1.  Basket  Assignment 

Assignment  of  a  population  to  baskets  follows  a  simple  set  of  rules: 

1.  All  the  proposals  in  the  population  are  arranged  in  descending  order  by  book 
value. 

2.  The  largest  unassigned  proposal  is  assigned  to  the  first  basket,  the  second 
largest  to  the  second  basket,  and  so  on  for  the  first  proposals. 

3.  The  baskets  are  sorted  in  ascending  order  by  total  assigned  proposal  book  value. 
(The  first  time  this  step  is  performed  it  will  simply  reverse  the  order  of  the 
baskets.) 

4.  The  assignment  and  sort  steps  are  repeated  until  all  the  proposals  are  assigned 
to  a  basket. 

The  effect  of  this  procedure  is  to  add  the  largest  of  the  next  n  unassigned 

proposals  to  the  basket  with  the  smallest  total  book  proposal  value.    The  end  result  is 

that  each  basket  will  have  nearly  identical  total  book  values  and  proposal  size  mixes. 

The  software  previously  noted  will  make  adjustments  to  correct  for  any  deviations  in 

totals  and  balance  the  mix  for  traits  other  than  cost  as  well. 
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2.  Evaluation  of  Baskets 

One  basket  is  chosen  randomly  and  all  its  proposals  are  audited.  The 
resulting  audit  value  of  the  basket  is  divided  by  the  proposal  value  to  produce  a 
correction  factor.  The  proposal  value  of  the  population  is  then  multiplied  by  the 
correction  factor  to  determine  the  population  audit  result.  This  will  be  the  estimaied 
true  value  of  the  population. 
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IV.  DESCRIPTION  OF  SIMULATION 

A.  DERIVING  COMPARABLE  RESULTS 

Direct  comparison  of  the  Basket  Method  and  the  Dollar  Unit  Sampling  method 
will  not  be  meaningful  since  the  former  yields  a  best  estimate  of  the  error  rate  of  a 
population  and  the  latter  returns  an  upper  error  limit  for  a  stated  assurance  level. 
Since  it  is  the  consistency  of  the  drawn  sample  which  is  of  interest  in  this  investigation, 
the  sample  selection  process  of  both  the  Basket  Method  and  the  Dollar  Unit  Sampling 
Technique  will  be  used  to  draw  samples  from  the  same  populations.  The  samples  will 
then  be  evaluated  according  to  the  Basket  Method,  which  will  give  an  estimate  of  the 
true  value  of  the  population,  to  see  how  well  the  sample  produced  by  each  method 
reflects  the  value  of  the  population.  This  is  appropriate  since  the  negotiated  price  of  a 
contract  should  be  a  best  estimate,  not  an  upper  limit. 

The  rules  of  the  Basket  Method  will  create  the  identical  set  of  baskets  from  a 
given  population  every  time  they  are  applied.  Therefore,  the  baskets  of  the  Basket 
Method  can  easily  be  evaluated  by  complete  enumeration.  The  rules  of  the  Dollar 
Unit  Sampling  technique  also  provide  a  finite  number  of  samples,  but  that  number  is 
much  larger  than  the  number  of  different  baskets.  Therefore,  the  Dollar  Unit  Sampling 
technique  will  be  evaluated  using  a  Monte  Carlo  process  in  which  a  random  start 
number  is  generated  for  each  run  and  the  AS  I  and  order  of  proposals  are  kept 
constant. 

To  evaluate  the  sample  drawn  with  the  Dollar  Unit  Sampling  rules  using  the 
Basket  Method  evaluation  technique  the  sample  is  treated  as  if  it  was  a  basket.  All  its 
resident  proposals  are  audited  and  their  true  value  is  divided  by  their  proposal  value. 
The  resulting  factor  is  multiplied  against  the  population  proposal  total  to  determine  the 
best  estimate  of  the  true  total  value  of  the  proposal  population. 

B.  PREPARATION  OF  THE  TEST  POPULATIONS 

The  populations  used  are  described  in  Table  1. 

The  original  populations  were  randomly  seeded  with  errors  at  a  5%  and  10% 
rate  of  occurance.  The  5%  population  was  then  skewed  to  form  two  additional  test 
populations.  One  had  its  errors  skewed  strongly  to  its  higher  valued  proposals,  the 
other  to  its  lower  valued  proposals.    The  total  dollar  amount  of  error  and  number  of 
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A 

B 

C 

D 

E 

8,300 

8,300 

8,300 

8,300 

8,300 

5% 

5% 

5% 

10% 

10% 

9% 

9% 

9% 

0.1% 

10% 

4- 

+ 

+ 

+ 

+ 

N 

H 

L 

N 

N 

AA 

BB 

CC 

DD 

EE 

8,300 

8,300 

8,300 

8,300 

8,300 

5% 

5% 

5% 

10% 

10% 

9% 

9% 

9% 

0.1% 

10% 

+  /■ 

+  /■ 

+  /- 

+  /- 

+  /■ 

N 

H 

L 

N 

N 

TABLE  1 
POPULATION  DESCRIPTION 

NAME 

Population 

Erroneous  Proposals 

£  S  Errors/^]  S  Proposals 

Types  of  Errors 

Skew  (none,  high,  or  low) 

NAME 

Population 

Erroneous  Proposals 

Y,  $  Errors/^]  $  Proposals 

Types  of  Errors 

Skew  (none,  high,  or  low) 

overstated  proposals  remained  constant  during  the  skewing  process.  All  populations 
were  created  in  both  a  "dishonest"  version  which  had  overstatements  only  and  are 
named  by  single  letters,  and  in  an  "honest"  version  with  both  overstatements  and 
understatements  named  by  double  letters.  Except  for  the  sign  on  each  error,  the  single 
letter  named  populations  are  identical  to  their  double  letter  named  counterparts.  The 
populations  with  10%  errors  differ  in  that  E  and  EE,  while  containing  the  same 
number  of  errors  in  the  same  distribution  and  sign  as  D  and  DD  respectively,  have 
errors  of  much  larger  magnitude,  so  that  the  sum  of  the  dollar  value  of  the  errors  make 
up  10%  of  the  population  in  E  and  EE  but  only  0.1%  of  the  population  in  D  and  DD. 

C.       SIMULATION  EXECUTION 

Simulations  were  run  on  all  populations  using  both  a  Basket  Method  evaluating 
program  and  a  Dollar  Unit  Sampling  monte  carlo  simulation  program.  First,  ten 
baskets  were  chosen  for  the  Basket  Method  to  remain  consistent  with  the  earlier 
arbitrary  decision  to  audit  one  tenth  of  the  proposals.  This  resulted  in  830  proposals 
per  basket.  Next,  an  ASI  was  then  selected  to  produce  an  average  of  830  proposals  in 
the  Dollar  Unit  Sampling  simulation,  and  ten  trials  were  run  using  the  Dollar  Unit 
Sampling  selection  method.  The  programs,  written  for  this  study  in  Waterloo  Basic, 
are  listed  in  Appendix  A. 
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The  ten  audit  results  for  each  method  were  then  divided  by  their  respective 
proposal  sums  to  derive  the  correction  factors.  These  correction  factors  were 
multiplied  by  the  sum  of  all  proposals  to  give  the  predicted  true  total  for  the 
population.  The  difference  between  the  predicted  and  the  actual  total  was  then  divided 
by  the  sum  of  all  proposals  to  give  a  percent  error  for  each  basket  and  trial.  The 
means  of  the  percent  errors  for  each  method  and  each  population  are  listed  in  Table  2. 

TABLE  2 
SIMULATION  RESULTS  (MEAN  %  ERROR) 


NAME 

A 

B 

C 

D 

E 

Basket  Method 

0.77 

1.03 

0.66 

0.14 

0.35 

DUS 

2.40 

-2.06 

2.75 

0.25 

4.75 

NAME 

AA 

BB 

CC 

DD 

EE 

Basket  Method 

0.99 

0.99 

1.16 

0.15 

1.21 

DUS 

0.13 

0.14 

-0.01 

-0.08 

-0.28 

Detailed  results  are  shown  in  Appendix  B.  A  positive  percent  error  represents  an 
overestimate  and  a  negative  number  represents  an  underestimate  error.  Using  the 
Basket  Method  to  draw  a  sample  was  always  more  accurate  against  the  overstatements 
only  padding  technique.  Using  the  Dollar  Unit  Sampling  technique  to  draw  the  sample 
was  more  accurate  when  the  proposals  contained  errors  in  both  directions. 

D.       RESISTANCE  TO  PROPOSAL  RIGGING 

In  order  to  benefit  from  the  potential  time  and  labor  savings  a  sampling  system 
offers,  the  sampling  technique  must  be  resistant  to  padding  schemes.  If  not,  a 
contractor  has  much  to  gain  by  manipulating  the  errors  in  his  proposals.  Therefore,  as 
mentioned  previously,  the  central  issue  is  not  necessarily  to  determine  which  of  these 
two  methods  is  the  most  accurate,  but  to  discover  which  one  benefits  padding  schemes 
least.  When  comparing  a  method's  performance  between  the  single  and  double  letter 
versions  of  a  population  it  can  be  seen  that  the  Basket  Method  is  essentially  as  strict  or 
stricter,  from  the  governments  view,  when  estimating  the  value  of  the  overstatement 
only  population  as  it  is  when  estimating  the  value  of  the  honest  population.  For 
example,  the  A  population,  which  was  rigged  with  overstatements  only,  was  evaluated 
by  the  Basket  Method  at  a  value  exceeding  its  actual  value  by  0.77%.    The  AA 
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population,  with  randomly  decided  over  and  understatements,  was  evaluated  at  a  value 
exceeding  its  true  value  by  0.99%.  If  these  two  populations  had  been  submitted,  the 
contractor  who  made  overstatements  only  would  have  received  0.22%  less  in  payment 
from  the  government.  In  other  words,  padding  one's  proposed  contract  changes  with 
overstatements  in  random,  low,  or  high  skewed  distributions  prior  to  submitting  them 
to  the  Basket  Method  for  evaluation  is  not  likely  to  raise  the  resulting  estimate  for  the 
population  but  is  instead  likely  to  lower  the  estimated  value.  On  the  other  hand,  the 
sample  drawn  with  Dollar  Unit  Sampling  allowed  the  overstatement  padded 
populations  a  significantly  larger  estimate  than  it  did  the  honest  populations. 

E.  EVALUATION 

Assuming  honest  contractors  are  as  likely  to  understate  as  overstate  their  costs 
and  dishonest  contractors  are  not,  honest  contractors  will  be  more  successful  than 
dishonest  contractors  against  the  Basket  Method,  and  dishonest  contractors  will  be 
more  successful  than  honest  contractors  against  the  Dollar  Unit  Sampling  sample 
method.  To  choose  the  Dollar  Unit  Sampling  method  to  determine  a  contract  value 
would,  in  effect,  encourage  overbidding,  since  it  would  be  more  profitable  than 
remaining  honest.    Under  the  Basket  Method  honesty  is  as  profitable  as  dishonesty. 

F.  AREAS  FOR  FURTHER  RESEARCH 

This  study  may  be  expanded  in  several  ways.  It  may  be  repeated  with  fewer  or 
more  baskets.  A  data  set  with  a  much  smaller  variance  in  proposal  size  may  be  used. 
The  Basket  Method  can  be  compared  to  additional  sampling  methods.  Additional 
strategies  for  error  arrangements  which  will  reduce  the  accuracy  of  the  Basket  Method 
can  be  developed  and  tested.  Particularly,  will  an  understatement  only  strategy  with 
the  same  magnitude  and  distribution  of  proposal  errors  result  in  a  total  population 
error  in  equal  magnitude  and  opposite  sign  as  an  overstatement  strategy. 
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APPENDIX  A 
SIMULATION  PROGRAM  LISTINGS 

Basket  Method  Enumeration  Program  Listing 

00100  REM  THIS  IS  A  PROGRAM  TO  PROCESS  DATA  USING  THE  BASKET  METHOD 

00120  REM  DATA  IS  INPUT  FROM  A  FILE.  SEPARATED  BY  COMMAS,  AND  LISTED 

00140  REM  AS  PAIRS  OF  VALUES  FOR  A  6lD.  THE  BOOK  FIRST  AND  THE 

00160  REM  AUDITED  VALUE  SECOND.   THE  PROGRAM  EXPECTS  'DATPOP'  PAIRS 

00180  REM  OF  VALUES.   DATA  MUST  BE  IN  DESCENDING  ORDER  BY  BOOK  VALUE. 

00200  REM 

00220  REM  **  DIMENSION  VARIABLES  ** 

00240  REM 

00260  DIM  ASUM[50),  BSUMf50),  ANEXT(50).  BNEXT(50) 

00280  DIM  ERRORP(50),  FACTOR(50),  ERRORA(50) 

00300  REM 

C0320  REM  **  SET  CONSTANTS  ** 

00340  REM 


00360  B  =  10 

00380  DATPOP  =  8300 


NUMBER  OF  BASKETS 
NUMBER  OF  DATA  PAIRS 


00400  BPOP=INT(DATPOP/B)  !  INITIATE  RUNNING  TALLY  OF  DATA  PAIRS  READ 

00420  OPEN  #3,  'TEST  (RECFM  F  LRECL  80)',  INPUT 

00440  ATOT  =  0 

00460  BTOT  =  0 

00480  BPOP1  =  1 

00500  FOR  J  = 

00520   ASUM(J 

00540   BSUM(J 

00560   NEXT  J 

00580  EES  =  0  !   SUM  OF  ERROR  SQUARES 

00600  EED  =  0  !   SUM  OF  BASKET  DOLLAR  SQUARES 

01000  REM 

01020  REM  **  ROUTINE  TO  READ  IN  DATA  ** 

01040  REM 

01060  IF  BPOP1  >  BPOP 

01080    GOTO  4000     !  IF  NO  MORE  DATA,  THEN  PROCESS 

01100     ENDIF 

01120  FOR  I  =  1  TO  B 

01140     INPUT  #3,  ANEXT(I),  BNEXT(I) 

01160     NEXT  I 

01180  BPOP1  =  BPOP1  +  1 

02000  REM 

02020  REM  **  ROUTINE  TO  SORT  PARTIAL  SUMS  IN  ** 

02040  REM  **    BASKETS  IN  ASCENDING  ORDER 

02060  REM 

02080  I  =  1 

02100  WHILE  I  <  B 

02120     IF  BSUM(I)  >  BSUM(I+1) 

02140       CI  =  BSUM(I) 

02160       C2  =  ASUM(I) 

02180  BSUM(I)  =  BSIMI+1) 

02200       ASUM(I)  =  ASUM(I+1) 

02220       BSUMfI+1)  =  CI 

02240       ASUM(I+1)  =  C2 

02260       IF  I>  1 

02280         I  =  1-1 

02300         ENDIF 

02320       GOTO  2120 

02340       ENDIF 

02360     I  =  1+1 

02380     ENDLOOP 

03000  REM 

03020  REM  **  ADD  NEXT  ROUND  TO  BASKETS  ** 

03040  REM 
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03060 
03080 
03100 
03120 
03140 
04000 
04020 
04040 
04060 
04080 
04100 
04120 
04140 
04160 
04180 
04185 
04190 
04200 
04220 
04240 
04260 
04280 
04300 
05000 
05020 
05040 
05060 
05080 
05100 
05120 
05140 
05160 
05180 
05200 
05220 
05240 
05260 
05280 
05300 
06000 
06020 
06040 
06060 
06080 
06100 
06120 
07000 


ADDING  ROUTINE 


1 


TO  B 
BTOT 
ATOT 

B 


BSUM(I) 
ASUM(I) 


FOR  I  =  1  TO  B 

BSUM(I)  =  BSUM(I)  +  BNEXT(I) 
ASUM(I)  =  ASUM(I)  +  ANEXT(I) 
NEXT  I 

GOTO  1060 
REM 

REM  ** 
REM 
FOR  I  = 

BTOT  = 

ATOT  = 

NEXT  I 
FOR  I  =  1 

FACTOR 

ERRORA 

EES  = 

EED  = 

ERRORP 

MAE  = 

MPE  =  MPE  + 

NEXT  I 
MAE  =  MAE  /  B 
MPE  =  MPE  /  B 
REM 

REM  **  PRINT 
REM 

PRINT  'BASKET 
FORMO$=rTOTAL 
F0RM1S='  ## 
F0RM2S='  MEAN 


=  ASUM(I)/BSUM(I) 

=  BTOT  *  FACTOR? I)  -  ATOT 

+  ERRORA(I)  *  ERRORA(I) 

+  BSUM(I)  *  BSUM(I) 

=  100  *  ERRORA(I)/BTOT 

+  ABS(ERRORA(I)) 

+  ABS(ERRORP(I)) 


RESULTS 


BOOK  VALUE 


AUDIT  VALUE 


FACTOR  %ERROR  ERR  OF  PR' 

#  ####  #####  ##' 
PRINT  USING  FORM0$,  BTOT,  ATOT,  ATOT/BTOT,  100*(BTOT-ATOT)/BTOT ,& 
&     BTOT  -  ATOT 
FOR  I  =  1  TO  B 

PRINT  USING  F0RM1$,  I,  BSUM(I),  ASUM(I),  FACTOR(I),  ERRORP(I),& 

NEXT  I 
PRINT  USING  F0RM2$,  MPE,  MAE 

F0RM3$=r     DOLLARS  #UMM.M  tiMMM.W 

F0RM4$= ■ CONTRACTS  #######. ##  #######. ## ' 

PRINT   '  MEAN  AUDITED  S.    D. ' 

PRINT  USING  F0RM3$,BT0T/B,SQR(((B*EED)-(BT0T*BT0T))/(B*(B-1))) 
PRINT  USING   F0RM4$  DATPOP;b;0 
REM 

REM     **     CLEANUP     ** 
REM 

CLOSE  #3 
END 
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Dollar  Unit  Sampling  Monte  Carlo  Program  Listing 


00010 
00100 
00120 
00140 
00160 
00180 
00200 
00220 
00240 
00260 
01000 
01020 
01040 
01060 
01080 
01100 
01120 
01140 
01160 
01180 
01200 
01220 
01240 
01260 
01280 
01300 
01310 
01320 
01340 
01360 
01380 
01400 
01420 
01440 
01460 
01480 
02000 
02020 
02040 
02060 
02080 
02100 
02120 
02140 
02160 
02180 
02200 
03000 
03020 
03040 
03060 
03080 
03100 
03120 
03140 
03160 
03280 
03300 
03320 
03340 
04000 
04020 
04040 
04060 
04080 
04100 
04120 


****** 


****** 


DUS  USING  ASI  ON  TESTSET 
THIS  IS  A  PROGRAM  TO  PROCESS  DATA  USING  DOLLAR  UNIT  SAMPLING. 
DATA  IS  INPUT  FROM  A  FILE.  SEPARATED  BY  COMMAS,  AND  LISTED 
AS  PAIRS  OF  VALUES  FOR  A  SlD.  THE  BOOK  FIRST  AND  THE 
AUDITED  VALUE  SECOND.   THE  PROGRAM  EXPECTS  'DATPOP'  PAIRS 
OF  VALUES.   DATA  MAY  BE  IN  ANY  ORDER 


REM 
REM 
REM 
REM 
REM 
REM 
REM 
REM 
REM 
RANDOMIZE 
REM 
REM 
REM 
L  =  0 
LI  =  1 
0 
0 
0 
0 
0 


SET  CONSTANTS 


RANDOMIZE  THE  START  OF  THE  RANDOM  NUMBER  GEN 


SET  THESE  VALUES  FOR  EACH  NEW  FILE 


'   LAP  COUNTER 
TOTAL  LAPS  TO 
SUM  OF  ACTUAL 


SUM 
SUM 
SUM 
SUM 


OF 
OF 
OF 
OF 


B 
(RECFM 


F  LRECL  80)',  INPUT 


! 


EEA 

EEP 

EES 

EEL 

EEV 

EC  =  0 

EC2  =  0 

ED  =  0 

ED2  =  0 

B  =  21 

DATPOP  =  8300 

ASI  =  DATPOP  / 

OPEN  #3,  'TEST 

POP  =  0 

AUDITSUM  =  0 

RESULTS  =  0 

BIDTOTAL  =  0 

Tn,UETOTAL=  0 

ECC  =  0 

K  =  RND  *  ASI 

RTALLY  =  -K 

REM 

REM  **  ROUTINE  TO  READ  IN  DATA 

REM 

POP  =  POP  +  1 

IF  POP  >  DATPOP 

GOTO  4000 

ENDIF 
INPUT  #3,  NEXTAUD, 
BIDTOTAL  =  BIDTOTAL  +  NEXTBID 
TRUETOTAL  =  TRUETOTAL  +  NEXTAUD 
RTALLY  =  RTALLY  +  NEXTBID 
REM 

REM  ** 
REM  ** 
REM 

IF  RTALLY 
GOTO  2060 
ENDIF 
R  =  CEIL(RTALLY/ASI) 
RTALLY  =  RTALLY  -  ( R 
AUDITSUM  =  AUDITSUM 
RESULTS  =  RESULTS  + 
ECC  =  ECC  +  1 
GOTO  2060 

REM  **  FINAL  CALCULATIONS  ** 
REM 

F  =  TRUETOTAL/BIDTOTAL 
AE  =  BIDTOTAL  -  TRUETOTAL 
FF  =  RESULTS  /  AUDITSUM 

=  (BIDTOTAL  *  FF)  -  TRUETOTAL 

=  EA  *  100  /  BIDTOTAL 


BE  RUN 
ERROR 
PERCENTAGE  ERROR 
ERROR  SQUARES 
PREDICTIONS 
AUDIT  SIZE  IN  DOLLARS 
NUMBER  OF  CONTRACTS  AUDITED 
EC-SQUARED 
TOTAL  BID  DOLLARS 
ED-SQUARED 

NUMBER  OF  BASKETS  USED  IN  ALT  METHOD 
NUMBER  OF  DATA  PAIRS 


IN  CONTRACTS  AUDITED 


IF  NO  MORE  DATA,  THEN  PROCESS 


NEXTBID 


ROUTINE  TO  DETERMINE  IF 
THIS  BID  WILL  BE  AUDITED 

<  0 


'   ASI) 
NEXTBID 
NEXTAUD 


EA 

EP 
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04140  EEA  =  EEA  +  EA 

04160  EEP  =  EEP  +  EP 

04180  EES  =  EES  +  EA  *  EA 

04200  EEL  =  EEL  +  BIDTOTAL  *  FF 

04220  EEV  =  EEV  +  AUDITSUM 

04240  EC  =  EC  +  ECC 

04260  EC2  =  EC2  +  (ECC  *  ECC) 

04280  ED  =  ED  +  AUDITSUM 

04300  ED2  =  ED2  +  (AUDITSUM  *  AUDITSUM) 

05000  REM 

05020  REM  **  PRINT  RESULTS  ** 

05040  REM 


05060  FORMOJ 

05080  F0RM1J 

05100  F0RM2' 

05101  FORM3< 


TOTAL     #MMM.M       #######.##       #.####  ###.###  ######.##' 
AUDIT     #######.##       #######.##       #.####  ###.###  ######.##' 

#######.##    #######.##   ######.###  #######.###' 

DOLLARS       #########.##       «######.## 
CONTRACTS       M#MMMM       #########.##' 


05102  F0RM4< 

05120  IF  L  =  0 

05140  PRINT  'BASKET  BOOK  VALUE  AUDIT  VALUE  FACTOR  %ERROR  ABS  ERROR1 

05160  PRINT  USING  FORMOS,  BIDTOTAL,  TRUETOTAL,  F,& 

05180  &  100*(AE)/BIDTOTAL,& 

05200  &     AE 

05220  ENDIF 

05240     PRINT  USING  F0RM1$,  AUDITSUM,  RESULTS,  FF,& 

05260  &  100  *  ((BIDTOTAL  *  FF)  -  TRUETOTAL)/BiDTOTAL,& 

05280  &  (BIDTOTAL  *  FF)  -  TRUETOTAL 

06000  REM 

06020  REM  **  CLEANUP  ** 

06040  REM 

06060  CLOSE  #3 

06080  L  =  L  +  1 

06100  IF  L  <  LI 

06120    GOTO  1320 

06140    ENDIF 

06160  PRINT  '  MEAN  PRED       S.  D.      MEAN  %  ERR  MEAN  ERROR1 

06180  PRINT  USING  F0RM2$,  EEL/LI,  SQR( EES/( Ll-1 ) ) , EEP/L1 ,  EEA/L1 

06190  PRINT 

06200  PRINT  '  MEAN  AUDITED      S.  D. ■ 

06220  PRINT  USING  F0RM3$,ED/L1 ,SQR(((L1*ED2)-(ED*ED))/(L1*(L1-1))) 

06240  PRINT  USING  F0RM4$,EC/L1  SQR( ( I Ll*EC2)-( EC*EC) )/( Ll*( Ll-1 ) ) ) 

06260  PRINT  vu      '   v     "  v   v    "' 

06280  PRINT  'HOW  MANY  MORE1; 

06300  INPUT  L2 

06320  LI  =  LI  +  L2 

06340  IF  L2  >  0 

06360    GOTO  1320 

06380    ENDIF 

07000  END 
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APPENDIX  B 
DETAILED  RESULTS 

POPULATION  A 


BASKET 

BOOK  VALUE 

:  AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

409605.73 

372255.73 

0.9088 

37350.00 

1 

40960.47 

36910.47 

0.9011 

-00.77 

-3150. 10 

2 

40960.51 

38170.51 

0.9319 

002.31 

9449.96 

3 

40960.57 

37450.57 

0.9143 

000.55 

2250.00 

4 

40960.57 

37540.57 

0.9165 

000.77 

3150.00 

5 

40960.55 

37270.55 

0.9099 

000.11 

449.  98 

6 

40960.56 

36370.56 

0.8879 

-02.09 

-8550.01 

7 

40960.57 

37090.57 

0.9055 

-00.33 

-1350.00 

8 

40960.66 

37270.66 

0.9099 

000.11 

450.08 

9 

40960.64 

37090.64 

0.9055 

-00.33 

-1349.94 

10 

40960.63 

37090.63 

0.9055 

-00.33 

-1349.95 

MEAN 

000.77 

3150.00 

MEAN 

AUDITED 

S.  D. 

DOLLAR!       409 

0.06 

CONTRACTS 

830.00 

0.00 

DUS 

BOOK  VALUE 

;  AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

409605.73 

372255.73 

0.9088 

37350.00 

AUDIT 

158976.31 

148626.31 

0.9349 

2.608 

10683.01 

AUDIT 

158949. 19 

148599. 19 

0.9349 

2.607 

10678.46 

AUDIT 

158824.04 

147574.04 

0.9292 

2.035 

8336.35 

AUDIT 

158604.  10 

148614. 10 

0.9370 

2.820 

11550. 15 

AUDIT 

158856. 17 

147966.17 

0.9314 

2.263 

9270.47 

AUDIT 

158996. 13 

149096. 13 

0.9377 

2.892 

11845.63 

AUDIT 

158693.67 

147443.67 

0.9291 

2.029 

8312.52 

AUDIT 

158827.85 

147757.85 

0.9303 

2.  149 

8801.26 

AUDIT 

158825.64 

147755.64 

0.9303 

2.149 

8800.86 

AUDIT 

159021.75 

148401.75 

0.9332 

2.440 

9995.  17 

MEAN 

PRED 

S.  D. 

MEAf 

MEAN  ERROR 

382083. 12 

10441.56 

2.399 

9827. 389 

MEAN 

AUDITED 

S.  D. 

DOLLARS     15! 

134.49 

CONTRACTS 

829.80 

0.42 

23 


POPULATION  B 


BASKET 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

409605.73 

372255.73 

0.9088 

37350.00 

1 

40960.52 

37270.52 

0.9099 

000.11 

449.95 

2 

40960.53 

36820.53 

0.8989 

-00.99 

-4050.04 

3 

40960.56 

37630.56 

0.9187 

000.99 

4049.99 

4 

40960.55 

37000.55 

0.9033 

-00.55 

-2250.02 

5 

40960.57 

36550.57 

0.8923 

-01.65 

-6750.00 

6 

40960.61 

36910.61 

0.9011 

-00.77 

-3149.96 

7 

40960.62 

37450.62 

0.9143 

000.55 

2250.04 

8 

40960.60 

38080.60 

0.9297 

002.09 

8550.02 

9 

40960.61 

36730.61 

0.8967 

-01.21 

-4949.96 

10 

40960.56 

37810.56 

0.9231 

001.43 

5849.99 

MEAN 

001.03 

4230.00 

MEAN 

AUDITED 

S.  D. 

DOLLARS 

0.04 

CONTRACTS 

830.00 

0.00 

DUS 

BOOK  VALUE 

:  AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

409605.73 

372255.73 

0.  9088 

37350.00 

AUDIT 

171892.21 

152812.21 

0.8890 

-1.981 

-8116. 15 

AUDIT 

171815.30 

152735.30 

0.8890 

-1.986 

-8136.50 

AUDIT 

172017.42 

151497.42 

0.8807 

-2.810 

-11511.97 

AUDIT 

171892.21 

152812.21 

0.8890 

-1.981 

-8116.15 

AUDIT 

172155.89 

153435.89 

0.8913 

-1.755 

-7189.98 

AUDIT 

172040.59 

152150.59 

0.8844 

-2.443 

-10005.44 

AUDIT 

171784.85 

153334.85 

0.8926 

-1.622 

-6642.39 

AUDIT 

172091.02 

152651.02 

0.8870 

-2.178 

-8920.49 

AUDIT 

172171.50 

153361.50 

0.8907 

-1.807 

-7400.05 

AUDIT 

171894.96 

152814.96 

0.8890 

-1.981 

-8115.42 

MEAN 

PRED 

S.  D. 

MEAf 

MEAN  ERROR 

363840.28 

8985.  13 

-2.055 

-8415.455 

MEAN 

AUDITED 

S.  D. 

DOLLAR!      1719: 

138.51 

CONTRACTS 

830.00 

0.00 

24 


POPULATION  C 


BASKET 

BOOK  VALUE 

:  AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

409605.73 

372255.73 

0.9088 

37350.00 

1 

40960.48 

37090.48 

0.9055 

-00.33 

-1350.09 

2 

40960.53 

37540.53 

0.9165 

000.77 

3149.96 

3 

40960. 57 

37000.57 

0.9033 

-00.55 

-2250.00 

4 

40960.54 

36910.54 

0.9011 

-00.77 

-3150.03 

5 

40960.57 

36910.57 

0.9011 

-00.77 

-3150.00 

6 

40960.58 

36910.58 

0.9011 

-00.77 

-3149.99 

7 

40960.59 

37720.59 

0.9209 

001.21 

4950.01 

8 

40960.64 

37180.64 

0.9077 

-00. 11 

-449.94 

9 

40960.64 

37630.64 

0.9187 

000.99 

4050.05 

10 

40960.59 

37360.59 

0.9121 

000.33 

1350.01 

MEAN 

000.66 

2700.01 

MEAN 

AUDITED 

S.  D. 

DOLLARS      409 

0.05 

CONTRACTS 

830.00 

0.00 

DUS 

BOOK  VALUE 

:  AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

409605.73 

372255.73 

0.9088 

37350.00 

AUDIT 

157773.16 

147783. 16 

0.9367 

2.787 

11414.28 

AUDIT 

157713.97 

148353.97 

0.9407 

3.  184 

13040.74 

AUDIT 

157738. 10 

148198. 10 

0.9395 

3.071 

12577.05 

AUDIT 

158027.14 

148217.  14 

0.9379 

2.911 

11922.52 

AUDIT 

157816.30 

147646.30 

0.9356 

2.674 

10954. 18 

AUDIT 

157838.59 

147398.59 

0.9339 

2.504 

10257.24 

AUDIT 

158054.38 

147794.38 

0.9351 

2.627 

10760.70 

AUDIT 

157816.61 

147556.61 

0.9350 

2.617 

10720.64 

AUDIT 

157782.93 

147702.93 

0.9361 

2.730 

11182.24 

AUDIT 

157840.97 

147310.97 

0.9333 

2.447 

10024.09 

MEAN 

PRED 

S.  D. 

MEAf 

MEAN  ERROR 

383541.10 

11935.61 

2.755 

11285.367 

MEAN 

AUDITED 

S.  D. 

DOLLARS      15; 

113.57 

CONTRACTS 

831.00 

0.00 
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POPULATION  D 


BASKET 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

375545.95 

372255.73 

0.9912 

3290.22 

1 

37503.29 

37274.23 

0.9939 

0.2653 

996.49 

2 

37507.89 

37282.36 

0.9940 

0.2748 

1032. 11 

3 

37520.51 

37172.52 

0.9907 

-.0513 

-192.84 

4 

37530.36 

37232.51 

0.9921 

0.0825 

309.80 

5 

37540. 11 

37167.81 

0.9901 

-.1156 

-434.22 

6 

37571.67 

37233.59 

0.9910 

-.0237 

-89.04 

7 

37580.30 

37271. 16 

0.9918 

0.0535 

200.93 

8 

37589.90 

37164.  13 

0. 9887 

-.2566 

-963.48 

9 

37591. 16 

37249.01 

0.9909 

-.0341 

-127.95 

10 

37610.76 

37208.41 

0.9893 

-.  1937 

-727.27 

MEAN 

0.1351 

507.41 

MEAN 

AUDITED 

S.  D. 

DOLLAR! 

38.68 

CONTRACTS 

830.00 

0.00 

DUS 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

375545.95 

372255.73 

0.9912 

0.876 

3290.22 

AUDIT 

154991.06 

154005.19 

0.9936 

0.240 

901.44 

AUDIT 

155019.49 

154034.41 

0.9936 

0.241 

903.79 

AUDIT 

154681.74 

153707.70 

0.9937 

0.246 

925.39 

AUDIT 

154986.95 

154047.14 

0.9939 

0.270 

1012.98 

AUDIT 

155019.33 

154032.08 

0.9936 

0.239 

898.53 

AUDIT 

154677.71 

153762.34 

0.9941 

0.284 

1067.77 

AUDIT 

154698.02 

153716.51 

0.9937 

0.242 

907.50 

AUDIT 

155015.48 

154063.07 

0.9939 

0.262 

982.88 

AUDIT 

154917.86 

153974.62 

0.9939 

0.267 

1003.65 

AUDIT 

154721.97 

153729.01 

0.9936 

0.234 

880.08 

MEAN 

PRED 

S.  D. 

MEAf 

MEAN  ERROR 

373204.13 

1001.71 

0.253 

948.401 

MEAN 

AUDITED 

S.  D. 

DOLLAR! 

156.44 

CONTRACTS 

838.  10 

0.32 
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POPULATION  E 


BASKET 

BOOK  VALUE 

:  AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

413614.33 

372255.73 

0.9000 

41358.60 

1 

41361.43 

37265.53 

0.9010 

000.  10 

399.60 

2 

41361.45 

36965.85 

0.8937 

-00.63 

-2597.38 

3 

41361.39 

37215.54 

0.8998 

-00.02 

-99.94 

4 

41361.39 

36765.99 

0.8889 

-01.11 

-4595.45 

5 

41361.39 

37265.49 

0.9010 

000.10 

399.56 

6 

41361.37 

37465.27 

0.9058 

000.58 

2397.54 

7 

41361.40 

37365.40 

0.9034 

000.34 

1398.57 

8 

41361.50 

37365.50 

0.9034 

000.34 

1398.66 

9 

41361.50 

37315.55 

0.9022 

000.22 

899.17 

10 

41361.51 

37265.61 

0.9010 

000.10 

399.68 

MEAN 

000.35 

1458.55 

MEAN 

AUDITED 

S.  D. 

DOLLARS      413i 

0.05 

CONTRACTS 

830.00 

0.00 

DUS 

BOOK  VALUE 

:  AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

413614.33 

372255.73 

0.9000 

41358.60 

AUDIT 

155856.65 

147964.55 

0.9494 

4.936 

20414.45 

AUDIT 

155788.89 

147746.94 

0.9484 

4.837 

20007.49 

AUDIT 

155686.91 

146995.61 

0.9442 

4.417 

18268.37 

AUDIT 

156020.28 

147578.73 

0.9459 

4.589 

18979.80 

AUDIT 

155820. 19 

147678.34 

0.9477 

4.774 

19746.60 

AUDIT 

155941.85 

148299.50 

0.9510 

5.099 

21088.32 

AUDIT 

155747.96 

147306.41 

0.9458 

4.579 

18940.68 

AUDIT 

155728.97 

146937.77 

0.9435 

4.354 

18009.28 

AUDIT 

155957.70 

148265.40 

0.9507 

5.067 

20957.90 

AUDIT 

155987.63 

147995.63 

0.  9488 

4.876 

20167. 14 

MEAN 

PRED 

S.  D. 

MEAf 

MEAN  ERROR 

391913.73 

20749.01 

4.753 

19658.002 

MEAN 

AUDITED 

S.  D. 

DOLLARS     15' 

117.46 

CONTRACTS 

830.50 

0.53 
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POPULATION  AA 


BASKET 

BOOK  VALUE 

:  AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

409605.73 

408975.73 

0.9985 

630.00 

1 

40960.47 

41050.47 

1.0022 

0.3735 

1530.00 

2 

40960.51 

40330.51 

0.9846 

-1.3843 

-5670.01 

3 

40960.57 

40690.57 

0.9934 

-.5054 

-2070.00 

4 

40960.57 

39880. 57 

0.9736 

-2.4829  ■ 

-10170.00 

5 

40960.55 

41410.55 

1.0110 

1.2524 

5130.00 

6 

40960.56 

41410.56 

1.0110 

1.2524 

5130.00 

7 

40960.57 

40870.57 

0.9978 

-.0659 

-270.00 

8 

40960.66 

41590.66 

1.0154 

1.6919 

6929.99 

9 

40960.64 

41050.64 

1.0022 

0.3735 

1530.00 

10 

40960.63 

40690.63 

0.9934 

-.5054 

-2070.00 

MEAN 

0.9888 

4050.00 

MEAN 

AUDITED 

S.  D. 

DOLLARS      409 

0.06 

CONTRACTS 

830.00 

0.00 

DUS 

BOOK  VALUE 

;  AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

409605.73 

408975.73 

0.9985 

630.00 

AUDIT 

158582.04 

158582.04 

1.0000 

0.154 

630.00 

AUDIT 

158560.98 

159100.98 

1.0034 

0.494 

2024.97 

AUDIT 

158265.77 

157905.77 

0.9977 

-.074 

-301.71 

AUDIT 

158495.01 

158855.01 

1.0023 

0.381 

1560.36 

AUDIT 

158599.40 

159049.40 

1.0028 

0.438 

1792. 19 

AUDIT 

158595.41 

159315.41 

1.0045 

0.608 

2489.55 

AUDIT 

158446.91 

158176.91 

0.9983 

-.017 

-67.98 

AUDIT 

158327.61 

157697.61 

0.9960 

-.244 

-999.86 

AUDIT 

158426.34 

157256.34 

0.9926 

-.585 

-2394.99 

AUDIT 

158583.44 

158493.44 

0.9994 

0.097 

397.54 

MEAN 

PRED 

S.  D. 

MEAf 

MEAN  ERROR 

409488. 74 

1610.08 

0.125 

513.006 

MEAN 

AUDITED 

S.  D. 

DOLLAR?      1584 

119.20 

CONTRACTS 

825.90 

0.32 
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POPULATION  BB 


BASKET 

BOOK  VALUE 

:  AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

409605.73 

410235.73 

1.0015 

-630.00 

1 

40960.52 

41230.52 

1.0066 

0.5054 

2070.00 

2 

40960.53 

40960.53 

1.0000 

-. 1538 

-630.00 

3 

40960.56 

41590.  56 

1.0154 

1.3843 

5670.00 

4 

40960. 55 

40600.55 

0.9912 

-1.0327 

-4230.00 

5 

40960.57 

40690.57 

0.9934 

-.8130 

-3330.00 

6 

40960.61 

40510.61 

0.9890 

-1.2524 

-5130.00 

7 

40960.62 

40330.62 

0.9846 

-1.6919 

-6929.99 

8 

40960.60 

41860.60 

1.0220 

2.0434 

8369.99 

9 

40960.61 

41410.61 

1.0110 

0.9448 

3870.00 

10 

40960.56 

41050.56 

1.0022 

0.0659 

270.00 

MEAN 

0.9888 

4050.00 

MEAN 

AUDITED 

S.  D. 

DOLLAR       409 

0.04 

CONTRACTS 

830.00 

0.00 

DUS 

BOOK  VALUE 

:  AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

409605.73 

410235.73 

1.0015 

-630.00 

AUDIT 

172018.41 

171208.41 

0.9953 

-.625 

-2558.75 

AUDIT 

171894.73 

172524.73 

1.0037 

0.213 

871.22 

AUDIT 

171856.38 

172126.38 

1.0016 

0.003 

13.52 

AUDIT 

172016.66 

171386.66 

0.9963 

-.520 

-2130.15 

AUDIT 

172023.91 

171753.91 

0.9984 

-.311 

-1272.90 

AUDIT 

172135.85 

174835.86 

1.0157 

1.415 

5794.78 

AUDIT 

171870.32 

172410.32 

1.0031 

0.  160 

656.94 

AUDIT 

171851.51 

172391.51 

1.0031 

0.160 

657.08 

AUDIT 

172035. 15 

171855. 15 

0.9990 

-.258 

-1058.57 

AUDIT 

172155.25 

174495.25 

1.0136 

1.205 

4937.52 

MEAN 

PRED 

S.  D. 

MEAf 

MEAN  ERROR 

410826.80 

2855.92 

0.144 

591.070 

MEAN 

AUDITED 

S.  D. 

DOLLAR*      1719! 

112.32 

CONTRACTS 

830.00 

0.00 
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POPULATION  CC 


BASKET 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

409605.73 

410595.73 

1.0024 

-990.00 

1 

40960.48 

40510.48 

0.  9890 

-1.3403 

-5490.01 

2 

40960.53 

41140.53 

1.0044 

0.  1978 

810.00 

3 

40960.57 

40600.57 

0.9912 

-1.1206 

-4590.00 

4 

40960.54 

41770.54 

1.0198 

1.7358 

7110.01 

5 

40960.57 

41590.57 

1.0154 

1.2964 

5310.00 

6 

40960.58 

41590.58 

1.0154 

1.2964 

5310.00 

7 

40960.59 

40240.59 

0.9824 

-1.9995 

-8190.00 

8 

40960.64 

40600.64 

0.9912 

-1.  1206 

-4589.99 

9 

40960.64 

41590.64 

1.0154 

1.2964 

5309.99 

10 

40960.59 

40960.59 

1.0000 

-.2417 

-990.00 

MEAN 

1.1645 

4770.00 

MEAN 

AUDITED 

S.  D. 

DOLLARS      409 

0.05 

CONTRACTS 

830.00 

0.00 

DUS 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

409605.73 

410595.73 

1.0024 

-990.00 

AUDIT 

157900. 17 

158170.17 

1.0017 

-.071 

-289.60 

AUDIT 

158128.98 

158218.98 

1.0006 

-.185 

-756.87 

AUDIT 

158031.95 

159291.95 

1.0080 

0.556 

2275.82 

AUDIT 

157685.03 

157595.03 

0.9994 

-.299 

-1223.79 

AUDIT 

158113.10 

158563. 10 

1.0028 

0.043 

175.76 

AUDIT 

157829.67 

157559.67 

0.9983 

-.413 

-1690.71 

AUDIT 

157795.92 

158155.92 

1.0023 

-.014 

-55.51 

AUDIT 

157782.24 

158232.24 

1.0029 

0.044 

178.21 

AUDIT 

157933.50 

158833.50 

1.0057 

0.328 

1344. 18 

AUDIT 

158021.11 

158291.11 

1.0017 

-.071 

-290. 13 

MEAN 

PRED 

S.  D. 

MEAf 

MEAN  ERROR 

410562.47 

1161.85 

-.008 

-33.265 

MEAN 

AUDITED 

S.  D. 

DOLLARS 

149.81 

CONTRACTS 

830.90 

0.32 
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POPULATION  DD 


BASKET 

BOOK  VALUE 

:  AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

372255.73 

372687. 17 

1.0012 

-431.44 

1 

37225.51 

37252.65 

1.0007 

-00.04 

-160.04 

2 

37225.53 

37351.67 

1.0034 

000.22 

829.96 

3 

37225.56 

37265.96 

1.0011 

-00.01 

-27.44 

4 

37225.55 

37097.64 

0.9966 

-00.46 

-1710.64 

5 

37225.57 

37253.81 

1.0008 

-00.04 

-149.04 

6 

37225.61 

37245.16 

1.0005 

-00.06 

-235.94 

7 

37225.61 

37347.20 

1.0033 

000.21 

784.46 

8 

37225.61 

37272.67 

1.0013 

000.01 

39.16 

9 

37225.61 

37382.72 

1.0042 

000.31 

1139.66 

10 

37225.56 

37217.69 

0.9998 

-00.  14 

-510.14 

MEAN 

000.15 

558.65 

MEAN 

AUDITED 

S.  D. 

DOLLAR! 

0.04 

CONTRACTS 

830.00 

0.00 

DUS 

BOOK  VALUE 

:  AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

372255.73 

372687.17 

1.0012 

-431.44 

AUDIT 

153449.50 

153530.55 

1.0005 

-.063 

-234.82 

AUDIT 

153678.01 

153708.72 

1.0002 

-.096 

-357.05 

AUDIT 

153419.97 

153541.36 

1.0008 

-.037 

-136.90 

AUDIT 

153377.53 

153443.02 

1.0004 

-.073 

-272.49 

AUDIT 

153713.33 

153672.20 

0.9997 

-.  143 

-531.05 

AUDIT 

153722.20 

153766.66 

1.0003 

-.087 

-323.78 

AUDIT 

153719.79 

153736.26 

1.0001 

-.105 

-391.56 

AUDIT 

153579.34 

153665.42 

1.0006 

-.060 

-222.79 

AUDIT 

153489.44 

153547.29 

1.0004 

-.078 

-291.  14 

AUDIT 

153472.87 

153584.51 

1.0007 

-.043 

-160.65 

MEAN 

PRED 

S.  D. 

MEAf 

MEAN  ERROR 

372394.95 

329.31 

-.079 

-292.222 

MEAN 

AUDITED 

S.  D. 

DOLLARS     1535 

136.34 

CONTRACTS 

831.30 

0.48 
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POPULATION  EE 


BASKET 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

413614.33 

416811.  13 

1.0077 

-3196.80 

1 

41361.43 

41061.73 

0.9928 

-1.4975 

-6193.80 

2 

41361.45 

40861.95 

0.9879 

-1.9805 

-8191.80 

3 

41361.39 

41910.84 

1.0133 

0.5555 

2297.71 

4 

41361.39 

42160.59 

1.0193 

1.  1593 

4795.21 

5 

41361.39 

41361.39 

1.0000 

-.7729 

-3196.80 

6 

41361.37 

41661.07 

1.0072 

-.0483 

-199.80 

7 

41361.40 

41960.80 

1.0145 

0.6763 

2797.20 

8 

41361.50 

40961.90 

0.9903 

-1.7390 

-7192.79 

9 

41361.50 

41910.95 

1.0133 

0.5555 

2297.69 

10 

41361.51 

42959.91 

1.0386 

3.0916 

12787.17 

MEAN 

1.2076 

4995.00 

MEAN 

AUDITED 

S.  D. 

DOLLARS      4131 

0.05 

CONTRACTS 

830.00 

0.00 

DUS 

BOOK  VALUE 

AUDIT  VALUE 

FACTOR 

%ERROR 

Numeric  Error 

TOTAL 

413614.33 

416811.13 

1.0077 

-3196.80 

AUDIT 

155935.54 

156584.89 

1.0042 

-.356 

-1474.42 

AUDIT 

155671. 11 

156320.46 

1.0042 

-.356 

-1471.49 

AUDIT 

155685.54 

156234.99 

1.0035 

-.420 

-1737.06 

AUDIT 

155821.71 

156321.21 

1.0032 

-.452 

-1870.92 

AUDIT 

155915.56 

156864.61 

1.0061 

-.  164 

-679.15 

AUDIT 

155840.41 

156339.91 

1.0032 

-.452 

-1871.08 

AUDIT 

155740.43 

157089.08 

1.0087 

0.093 

384.94 

AUDIT 

155955.47 

156005.42 

1.0003 

-.741 

-3064.33 

AUDIT 

155894.34 

156943.29 

1.0067 

-.  100 

-413.76 

AUDIT 

155769.62 

157168.22 

1.0090 

0.  125 

516.90 

MEAN 

PRED 

S.  D. 

MEAf 

MEAN  ERROR 

415643.09 

1659.85 

-.282 

-1168.038 

MEAN 

AUDITED 

S.  D. 

DOLLARS 

155822.97 

103.18 

CONTRACTS 

830.40 

0.52 
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